15 research outputs found
NSNet: A General Neural Probabilistic Framework for Satisfiability Problems
We present the Neural Satisfiability Network (NSNet), a general neural
framework that models satisfiability problems as probabilistic inference and
meanwhile exhibits proper explainability. Inspired by the Belief Propagation
(BP), NSNet uses a novel graph neural network (GNN) to parameterize BP in the
latent space, where its hidden representations maintain the same probabilistic
interpretation as BP. NSNet can be flexibly configured to solve both SAT and
#SAT problems by applying different learning objectives. For SAT, instead of
directly predicting a satisfying assignment, NSNet performs marginal inference
among all satisfying solutions, which we empirically find is more feasible for
neural networks to learn. With the estimated marginals, a satisfying assignment
can be efficiently generated by rounding and executing a stochastic local
search. For #SAT, NSNet performs approximate model counting by learning the
Bethe approximation of the partition function. Our evaluations show that NSNet
achieves competitive results in terms of inference accuracy and time efficiency
on multiple SAT and #SAT datasets
Chronosymbolic Learning: Efficient CHC Solving with Symbolic Reasoning and Inductive Learning
Solving Constrained Horn Clauses (CHCs) is a fundamental challenge behind a
wide range of verification and analysis tasks. Data-driven approaches show
great promise in improving CHC solving without the painstaking manual effort of
creating and tuning various heuristics. However, a large performance gap exists
between data-driven CHC solvers and symbolic reasoning-based solvers. In this
work, we develop a simple but effective framework, "Chronosymbolic Learning",
which unifies symbolic information and numerical data points to solve a CHC
system efficiently. We also present a simple instance of Chronosymbolic
Learning with a data-driven learner and a BMC-styled reasoner. Despite its
great simplicity, experimental results show the efficacy and robustness of our
tool. It outperforms state-of-the-art CHC solvers on a dataset consisting of
288 benchmarks, including many instances with non-linear integer arithmetics
TorchProbe: Fuzzing Dynamic Deep Learning Compilers
Static and dynamic computational graphs represent two distinct approaches to
constructing deep learning frameworks. The former prioritizes compiler-based
optimizations, while the latter focuses on programmability and
user-friendliness. The recent release of PyTorch 2.0, which supports compiling
arbitrary deep learning programs in Python, signifies a new direction in the
evolution of deep learning infrastructure to incorporate compiler techniques in
a more dynamic manner and support more dynamic language features like dynamic
control flows and closures. Given PyTorch's seamless integration with Python,
its compiler aims to support arbitrary deep learning code written in Python.
However, the inherent dynamism of Python poses challenges to the completeness
and robustness of the compiler. While recent research has introduced fuzzing to
test deep learning compilers, there is still a lack of comprehensive analysis
on how to test dynamic features. To address this issue, we propose several code
transformations to generate test cases involving dynamic features. These
transformations preserve the program's semantics, ensuring that any discrepancy
between the transformed and original programs indicates the presence of a bug.
Through our approach, we have successfully identified twenty previously unknown
bugs in the PyTorch compiler and its underlying tensor compiler Triton
Scalar Invariant Networks with Zero Bias
Just like weights, bias terms are the learnable parameters of many popular
machine learning models, including neural networks. Biases are thought to
enhance the representational power of neural networks, enabling them to solve a
variety of tasks in computer vision. However, we argue that biases can be
disregarded for some image-related tasks such as image classification, by
considering the intrinsic distribution of images in the input space and desired
model properties from first principles. Our findings suggest that zero-bias
neural networks can perform comparably to biased networks for practical image
classification tasks. We demonstrate that zero-bias neural networks possess a
valuable property called scalar (multiplication) invariance. This means that
the prediction of the network remains unchanged when the contrast of the input
image is altered. We extend scalar invariance to more general cases, enabling
formal verification of certain convex regions of the input space. Additionally,
we prove that zero-bias neural networks are fair in predicting the zero image.
Unlike state-of-the-art models that may exhibit bias toward certain labels,
zero-bias networks have uniform belief in all labels. We believe dropping bias
terms can be considered as a geometric prior in designing neural network
architecture for image classification, which shares the spirit of adapting
convolutions as the transnational invariance prior. The robustness and fairness
advantages of zero-bias neural networks may also indicate a promising path
towards trustworthy and ethical AI.Comment: 22 pages, 8 figure
Novice Type Error Diagnosis with Natural Language Models
Strong static type systems help programmers eliminate many errors without
much burden of supplying type annotations. However, this flexibility makes it
highly non-trivial to diagnose ill-typed programs, especially for novice
programmers. Compared to classic constraint solving and optimization-based
approaches, the data-driven approach has shown great promise in identifying the
root causes of type errors with higher accuracy. Instead of relying on
hand-engineered features, this work explores natural language models for type
error localization, which can be trained in an end-to-end fashion without
requiring any features. We demonstrate that, for novice type error diagnosis,
the language model-based approach significantly outperforms the previous
state-of-the-art data-driven approach. Specifically, our model could predict
type errors correctly 62% of the time, outperforming the state-of-the-art
Nate's data-driven model by 11%, in a more rigorous accuracy metric.
Furthermore, we also apply structural probes to explain the performance
difference between different language models.Comment: 17 pages, 8 figure
Effective Interactive Resolution of Static Analysis Alarms
We propose an interactive approach to resolve static analysis alarms. Our approach synergistically combines a sound but imprecise analysis with precise but unsound heuristics, through user interaction. In each iteration, it solves an optimization problem to find a set of questions for the user such that the expected payoff is maximized. We have implemented our approach in a tool, Ursa, that enables interactive alarm resolution for any analysis specified in the declarative logic programming language Datalog. We demonstrate the effectiveness of Ursa on a state-of-the-art static datarace analysis using a suite of 8 Java programs comprising 41-194 KLOC each. Ursa is able to eliminate 74% of the false alarms per benchmark with an average payoff of 12 per question. Moreover, Ursa prioritizes user effort effectively by posing questions that yield high payoffs earlier
Optimization and Abstraction: A Synergistic Approach for Analyzing Neural Network Robustness
In recent years, the notion of local robustness (or robustness for short) has
emerged as a desirable property of deep neural networks. Intuitively,
robustness means that small perturbations to an input do not cause the network
to perform misclassifications. In this paper, we present a novel algorithm for
verifying robustness properties of neural networks. Our method synergistically
combines gradient-based optimization methods for counterexample search with
abstraction-based proof search to obtain a sound and ({\delta}-)complete
decision procedure. Our method also employs a data-driven approach to learn a
verification policy that guides abstract interpretation during proof search. We
have implemented the proposed approach in a tool called Charon and
experimentally evaluated it on hundreds of benchmarks. Our experiments show
that the proposed approach significantly outperforms three state-of-the-art
tools, namely AI^2 , Reluplex, and Reluval
Learning-Aided Program Synthesis and Verification
The enormous rise in the scale, scope, and complexity of software projects has created a thriving marketplace for program reasoning tools. Despite broad adoption by industry, developing such tools remains challenging. For each project, specialized heuristics or analysis rules have to be carefully designed and customized, which requires non-trivial expertise. Recently machine learning, especially deep learning, achieved remarkable successes in many challenging areas such as image recognition and strategy game playing. Inspired by these successes, this thesis is concerned with the following question: can program reasoning be effectively learned and automatically improved over time? This thesis demonstrates that learning-based techniques can be a new driving force for tackling fundamental program reasoning challenges, particularly, program synthesis and program verification. First, this thesis presents a scalable inductive logic programming (ILP) framework, Difflog, which can synthesize a rich set of logical rules used in various important domains like program analysis, relational query and knowledge discovery. Unlike classic program synthesis techniques, which heavily rely on manually designed heuristics or symbolic constraint solvers, Difflog leverages efficient gradient-based approaches, which is possible due to a novel numerical relaxation of logical rules. Second, this thesis presents an end-to-end deep learning framework for program verification, Code2Inv, which directly maps a piece of source code to its related proof without requiring any annotations from human experts. Code2Inv is inspired by the recent AI breakthrough, AlphaGo; however, unlike the two-dimensional game board, programs have sophisticated structures and correct proofs are extremely rare, posing unique challenges on representation learning and reinforcement learning. To address these challenges, we leverage advances of graph neural networks and develop a counterexample-based smooth reward mechanism. Code2Inv outperforms state-of-the-art approaches that are based on manually designed heuristics or decision tree learning, and the learned policy by Code2Inv can generalize to unseen programs. Furthermore, Code2Inv can be flexibly customized as a Constrained Horn Clause (CHC) solver as well as a meta-solver for syntax-guided program synthesis tasks